1. Field of the Invention
This invention relates to image data processing. Particularly, this invention relates to image data processing for automated recognition and sensor control on space platforms.
2. Description of the Related Art
Gigapixel mosaic focal planes have been proposed for very wide field-of-view (VWFOV) imaging with high resolution at the region of interest (ROI) level to detect and track sparse events within the VWFOV. In order to capture such events using a gigapixel focal plane array without dynamic adaptation, immense volumes of data need to be post processed to identify and track such events through an image data space, e.g. consisting of terabytes of data. This methodology is not only time consuming but may also result in missing significant events that require real-lime adaptation of the focal plane array (e.g. pointing, focal plane operating conditions) to observe the phenomena with the required level of fidelity.
Imaging for intelligence/surveillance (e.g., from spacecraft) is typically directed to collecting data from space on observable phenomena of potential intelligence value occurring anywhere on earth and at anytime. In addition, it may involve processing the collected data into information and communicating and disseminating that information as actionable intelligence. Such intelligence data is often dominated by high-resolution imagery due to the emergence of large format digital cameras and can create a tactical advantage through rapid dissemination. However, one limitation is that data link bandwidths from sensor platforms in space are limited to hundreds of megabits per second. Given the extended distances required for such links, there is no near term solution to realizing a data link or network technology that can reasonably match the rate at which large format digital cameras collect data. This problem may be further exacerbated with sensor technologies that move into the realm of gigapixel imagery.
Thus, a primary challenge in realizing a gigapixel sensor capable of in-situ situational awareness is the need for on-the-fly automated processing of large volumes of streaming data with native intelligence to filter the raw data and update the sensor state to focus on the truly useful data within the scene. Due to data link bandwidth constraints, a lack of in-situ intelligence will result in low-to-medium resolution imagery or reduced acquisition rates and resultant limits on detection and recognition capabilities.
A new approach is required to create the possibility of real time perception, and recognition that include identification, tracking, and response to multiple moving targets. This paradigm requires converting the processed results into meaningful sensor control information in real time to keep pace with changes in the scene imagery by eliminating data having little informational value and retaining only data with pertinent information. The challenge of such an adaptive mechanism is not only on-the-fly processing but also system-level coordination for rapidly turning around the results to achieve dramatic data reduction and a reduced computational load.
Some current technologies offer real-time processing of data using parallel computational environments running on general purpose multi-processors, e.g., Parallel Computational Environment for Imaging Science (PiCEIS), or operator-controlled foveation within a gigapixel array camera for manual target tracking, e.g., ResoLUT gigapixel camera. PiCEIS is an image processing package designed for efficient execution on massively parallel computers and has demonstrated 10-second processing time for very large image files. ResoLUT is a gigaPixel surveillance camera with selective foveation. However, the PiCEIS environment requires massive bandwidth from the sensor to the mainframe system to achieve the image processing time reported for this work and the ResoLUT camera has the low-level control built into the camera to provide for on-the-fly adaptation of the camera but lacks integrated processing and control needed to update the camera operational state based on image dynamics. Such existing conventional systems are limited to a resolution of one gigapixel and require back end processing and direct user control to operate. Additionally, current systems are designed to accept and process data from at most two imagers for stereoscopic applications and lack the scalability for multi-gigapixel mosaic focal plane arrays.
In view of the foregoing, there is a need in the art for apparatuses and methods for high volume image data processing. There is a need for such apparatuses and methods for long-range surveillance applications due to their high spatial resolution and wide field-of-view. There is a need for such systems and methods for automated image recognition and sensor control. There is particularly a need for such apparatuses on space platforms due to limited data link bandwidths for remote recognition and control. These and other needs are met by the present invention as detailed hereafter.